2,009 research outputs found
CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluation
Practical dialog systems need to deal with various knowledge sources, noisy
user expressions, and the shortage of annotated data. To better solve the above
problems, we propose CGoDial, new challenging and comprehensive Chinese
benchmark for multi-domain Goal-oriented Dialog evaluation. It contains 96,763
dialog sessions and 574,949 dialog turns totally, covering three datasets with
different knowledge sources: 1) a slot-based dialog (SBD) dataset with
table-formed knowledge, 2) a flow-based dialog (FBD) dataset with tree-formed
knowledge, and a retrieval-based dialog (RBD) dataset with candidate-formed
knowledge. To bridge the gap between academic benchmarks and spoken dialog
scenarios, we either collect data from real conversations or add spoken
features to existing datasets via crowd-sourcing. The proposed experimental
settings include the combinations of training with either the entire training
set or a few-shot training set, and testing with either the standard test set
or a hard test subset, which can assess model capabilities in terms of general
prediction, fast adaptability and reliable robustness.Comment: EMNLP 202
Commencement Program, May (1987)
https://red.mnstate.edu/commencement/1145/thumbnail.jp
- …